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Summary 


The primary goal of the U.S. Department of Education’s Institute of Education Sciences (IES) 2012-17 
Regional Educational Laboratory (REL) contract cycle was to “help states and districts systematically use 
data and analysis to answer important issues of policy and practice with the goal of improving student 
outcomes.” The most notable change from previous REL contract cycles was the requirement that RELs 
focus at least 85 percent of their resources on technical assistance and research-related projects associated 


with REL research alliances. 


This report provides a snapshot of the variations and experiences across the 10 RELs during the 2012-17 
cycle’s effort to establish and support research alliances—an effort larger than any other prior research- 
practice partnership effort to date. It is not a formal evaluation of the REL research alliance program, nor 
is ita “how to guide.” Rather, this report adds to the growing literature base on research-practitioner 
partnerships by sharing how the RELs reported creating, engaging, and maintaining multiple partnerships, 
with the purpose of informing future collaborative efforts for researchers and practitioners and for those 


who wish to support research-practice partnerships. 


Highlights of the REL 2012-17 research alliance experience 


Coburn, Penuel, and Geil (2013, p. 2) define research-practice partnerships within and across school 
districts as “long-term, mutualistic partnerships between practitioners and researchers that are 
intentionally organized to investigate problems of practice and solutions for improving district outcomes.” 
REL research alliances contained several important attributes present in traditional research-practice 
partnerships, including defined roles between researchers and practitioners; collaboration on the research 
agenda setting and refinement; and designing, developing, testing, or refining educational improvement 


approaches in real-world contexts. 


Seventy-nine REL research alliances operated by year 4 of the contract cycle, between 6 and 10 research 
alliances per REL. These research alliances varied in origin, geographical spread, topic area, and diversity 


in membership. 


Most REL research alliances were newly formed. By the start of year 4, 70 of the 79 year 4 REL research 
alliances (89 percent) were newly created. That is, the alliance did not exist as a formal group before the 


REL helped to create it with regional partners. 


REL research alliances predominantly included multiple school districts within or across states. At the 
start of year 4, 35 of the 79 alliances (44 percent) included members across multiple states, 36 alliances (46 
percent) operated statewide or across multiple districts within a state, and 8 alliances (10 percent) included 


members from one school district or a small subset of districts within a state. 


REL research alliances covered a wide range of education topics. REL research alliances addressed the 
following topics: increasing high school completion and college readiness, access, and completion (25 


alliances); identifying and retaining effective teachers and principals (21 alliances); improving low- 


achieving schools (21 alliances); adopting and implementing rigorous academic standards and assessments 
(6 alliances); and early childhood education (6 alliances). 


Membership composition varied across research alliances within RELs. Member participation across 
alliances ranged from 4 to more than 250 individuals (median = 25) per REL research alliance. Average 
“active” membership ranged from 4 to 100 members, with the majority (56 percent) having 15 or fewer 
active members. Only 10 percent had more than 50 active members. Most members were district- or state- 


level staff, and the composition of membership varied by topic area of the alliances. 


RELs faced a variety of challenges and developed strategies to support research alliances 


RELs reported facing a wide range of challenges typically experienced by other research-practice 


partnerships. RELs implemented numerous strategies to address these challenges: 


e Building trust, engagement, and a sense of partnership. REL research alliances took advantage, 
when possible, of previously existing relationships. They also utilized REL staff who had 
background in navigating the research-practice divide, encouraged alliance members to take on 


leadership roles, and worked hard to foster a core team of engaged alliance members. 


e Fostering communication. RELs reported using creative and flexible methods to facilitate early 
and ongoing in-person and virtual meetings, and supplemented formal communication with as- 
needed discussions. REL research alliances also communicated regularly with nonalliance members 
and key stakeholders, for example, through hosting dissemination events to broader communities. 
Specifically chosen staff facilitated the exchange of information, which was communicated in a 


variety of ways such as presentations, archived videos, and infographics. 


e Managing time constraints. REL research alliances factored in time constraints when determining 
who would take on key leadership roles and continually recruited new members to replace those 
who had insufficient time. RELs found creative ways to support members when there were time 
lags in dissemination of findings, for example, by disseminating existing evidence-based resources or 
providing targeted technical assistance on emerging knowledge on specific topics. They developed 


and, when necessary, revised timelines to be realistic and responsive to the needs of stakeholders. 


e Working with research alliance members who have different priorities, interests, and 
expectations. REL research alliances worked continually to find common ground. They engaged in 
regular reflective work, sometimes by regularly recommitting to the research agenda. RELs 
developed strategies to identify and foster positive group dynamics. They acknowledged the need to 
be flexible in response to changes in state and local leadership priorities. In some cases, when needs 
were too disparate, they reconfigured alliances to cover smaller geographic regions and developed 
cross-state opportunities to facilitate the sharing of resources of common interest. In other cases, 
RELs developed smaller workgroups within an alliance to focus on issues that were of particular 


interest to some communities or members. 


e Building capacity. RELs often worked with research alliance members and their respective 


organizations or agencies to develop toolkits and targeted technical assistance activities that 
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supported specific needs within the larger goals of the alliance and to increase research capacity 
among practitioner agencies and organizations. At times, they used locally based staff to help build 
capacity. 
REL research alliances were further supported through cross-REL communication and collaboration 
throughout the 2012-17 REL contract cycle, including a working group specifically focused on research 
alliances. This working group met quarterly to discuss shared experiences and challenges and enabled 


RELs to learn from each other’s experiences to better serve the membership of their own alliances. 


The REL 2012-17 contract cycle was responsible for the implementation of the largest research-practice 
partnership effort in preK-12 education to date. This effort spanned all regions under jurisdiction of the 
U.S. and territories and included more than 70 distinct research alliances at any point during the 5-year 
contract cycle. While the RELs reported facing numerous challenges, they developed strategies to meet 
these challenges and support growth, which may be instructive for newly formed as well as existing 
research-practice partnerships. As research-practice partnerships continue to grow in popularity, it will be 
important to document their experiences, explore commonalities and differences, and share lessons 


learned. 


Chapter 1. Documenting the 2012-17 REL research alliance experience 


Demand is high for evidence-informed decisionmaking in education policy and practice. To provide 
evidence for informed decisions, education research should be relevant to policies and responsive to the 
needs of educators and policymakers.’ Researchers who understand their educational environment can 
better frame and carry out relevant and responsive studies. Collaborations—or partnerships—between 
researchers and practitioners can inform such studies and help bridge the gap between research and 
practice.’ Federal and state policymakers often have high regard for research findings from partnerships 
because of their local relevancy (Coburn & Penuel, 2016; Coburn et al., 2013). Similarly, school district- 
level decisionmakers often consider local information and findings to be the most useful research evidence 
(Honig and Coburn, 2008; Nelson, Leffler, @ Hansen, 2009). In turn, researchers who engage in work 
with practitioners help build the broader knowledge base, while also ensuring that their work is practically 


relevant. 


Partnerships can have many forms, vary in duration, and require participants to play a variety of roles. In 
its most basic form, a partnership could consist of practitioners, such as school district personnel, who 
work with a research consultant or local university professor to answer a practice-relevant research 
question. In contrast, another partnership might have a long-term, evolving research agenda that requires 
a group of researchers and practitioners to make a sustained commitment that could last for more than a 
decade. Coburn et al. (2013, pp. 2-4) define research-practice partnerships within and across school 
districts as “long-term, mutualistic partnerships between practitioners and researchers that are 
intentionally organized to investigate problems of practice and solutions for improving district 
outcomes. ... Research-practice partnerships go beyond the focus of many current organizations on 
making data accessible to district leaders.’ The partnerships instead produce original analyses of data to 
answer research questions posed by the district.” Although variation exists, Coburn et al. (2013) discuss 
how the different types of education-related research-practice partnerships share six common 
characteristics: they (1) are place-based, (2) are long-term, (3) focus on problems of practice, (4) use 
intentional strategies to foster partnership, (5) produce original analyses, and (6) are committed to 
“mutualism,” which the authors describe as “sustained interaction that benefits both researchers and 


practitioners.” 


Because of the growing interest in finding ways to bridge the gap between research and practice and to 


make research more relevant to practitioners, the education community is documenting how different 


: Throughout this report, the terms “educator(s)” and “practitioner(s)” are used synonymously. Also, the term 
“decisionmaker(s)” is used interchangeably with “policymaker(s).” 

* In the literature, these relationships are sometimes called research-practice partnerships and sometimes called researcher- 
practitioner collaborations. These relationships can take on a variety of forms and mean different things to different people. For 
the purposes of this report, “partnerships” and “collaborations” are used interchangeably. Further, the term “research 
alliances,” or “alliances” for short, is the term used during the 2012-17 REL contract cycle to refer to “partnerships” or 
“collaborations” led by the 10 RELs. 

> Coburn et al. (2013, p. 2) define long-term as “open-ended commitments [involving] more than a single consulting agreement 
or grant. The work can span a few years, or, as is true for some partnerships, more than a decade, shifting focus as the work 
develops over time.” 


research-practice partnerships operate, the challenges that they face, and the strategies that they use most 
effectively. This report adds to that literature (Bryk, Gomez, & Grunow, 2010; Coburn et al., 2013) by 
documenting the experiences of the research alliances that were formed during the 2012-17 contract cycle 
of the Regional Educational Laboratory (REL) program. This report is intended for those who are 


interested in engaging in similar research-practice partnerships. 


The REL program and the formation of research alliances 


The REL program, originally established as part of the 1965 Elementary and Secondary Education Act, 
aims to bridge the gap between research and practice. The REL program has updated its priorities and 
evolved over the past 50 years, but it has not wavered in its aim to make research responsive to the needs 
of practitioners. The current iteration of the REL program, as authorized by the Education Sciences 
Reform Act of 2002, includes 10 Regional Educational Laboratories* (“regional labs” or RELs) that serve 
the needs of their respective geographic regions across the United States and its territories.’ Consistent 
with the “central mission and primary function” of the REL program, each REL must “support applied 
research, development, wide dissemination, and technical assistance activities,” including “providing 
training... and technical assistance to state educational agencies, local educational agencies, school 
boards, schools funded by the Bureau as appropriate, and state boards of education... .”° The U.S. 
Department of Education’s Institute of Education Sciences (IES) contracts with organizations to operate 
each REL through a competitive bidding process, with each REL contract cycle running for a 5-year 
period. The primary goal of the 2012-17 REL contract cycle was to “help states and districts systematically 
use data and analysis to answer important issues of policy and practice with the goal of improving student 
outcomes.”’ The most notable change from previous cycles was the use of the research alliance structure in 


framing each REL’s technical assistance and research agenda. 


IES defines a REL research alliance as “a group of stakeholders who share a specific education concern 
and agree to work together to learn more about the concern so that they can make sound decisions to 
improve education outcomes.”® As a new feature of the 2012-17 contract cycle, IES required that RELs 
focus at least 85 percent of their resources on technical assistance and research-related projects associated 


with REL research alliances.” Each REL was tasked with helping to “maintain and refine research 


* See http://ies.ed.gov/ncee/edLabs/regions/. 


> The Regional Educational Laboratories program is authorized under the Education Sciences Reform Act of 2002, Part D, 
Section 174, (20 U.S.C. 9564) and administered by the Institute of Education Sciences’ National Center for Education 
Evaluation and Regional Assistance. See http://www.ed.gov/policy/rschstat/leg/PL107-279.pdf. For additional information 
about the legislation and regulations, see http://www2.ed.gov/programs/regionallabs/legislation.html. 

® Education Sciences Reform Act of 2002 (H.R. 3801) Part D, Section 174, National Center for Education Evaluation and 
Regional Assistance, Sec. 174, (20 U.S.C. 9564) (f). Regional educational laboratories for research, development, dissemination, 
and technical assistance. Central Mission and Primary Function. See http://www.ed.gov/policy/rschstat/leg/PL107-279.pdf. 
The term “Bureau” in the quoted passage from page 28 of H.R. 3801 refers to the Bureau of Indian Affairs. 


” See the “Performance Work Statement for Regional Educational Laboratories (2012-2017),” page 3; available at 
https://www.fbo.gov/utils/view?id=50688bfda3cc31d77469838a79649491. 

* Thid. 

” Ibid. 


alliances” by organizing meetings of and communications among members and by documenting and 
assessing, on an ongoing basis, the development of alliances. Together with REL research staff, alliance 
members identify local or regional issues, carry out relevant research, disseminate findings and products, 


and build their capacity to understand and use research and data. 


As of January 2015, the beginning of year 4 of the REL The 10 RELs supported 79 research alliances 
contract cycle, when the research alliances were in a mature in 2015. 


state, the 10 RELs supported 79 research alliances.'° Appendix 

A provides a list of the research alliances discussed in this report. Throughout the 2012-17 contract cycle, 
IES offered flexibility for RELs to tailor the optimal number of alliances as well as the structure, size, and 
focus of each alliance to the needs of their particular regions. IES encouraged RELs to form regional, 
cross-state, statewide, cross-district, or single-district research alliances where appropriate; the alliances 
could include state-level members, district-level members, or both.'’ Some research alliances also included 
university faculty, staff from nonprofits or governmental agencies, or other interested stakeholders within 
the region. A REL research alliance could be newly created, or the REL could form an alliance by joining 


an existing group or partnership of practitioners. 


The goal of this report 


This report describes the variations and experiences among the 10 RELs in their creation, maintenance, 
and support of research alliances. The 2012-17 REL program’s effort to establish and support research 
alliances was larger than any other prior research-practice partnership effort.'? The goal of this report is to 
provide a snapshot of that 2012-17 effort. The report synthesizes information collected from the RELs by 
IES to document the REL research alliance experience. This report is not intended as an evaluation of the 
REL research alliance effort, not is it a substitute for the ongoing reflective efforts that RELs have 
undertaken to document and analyze their respective experiences.'’ By taking a broad view of all REL- 


supported research alliances, this report: 


e Adds to the growing literature base on research-practitioner partnerships; 


m By year 4, it was unlikely that new alliances would develop or that existing alliances would change significantly. If the authors 
of this report had waited until the final year (year 5) to assess the REL research alliance evolution, then the report would have 
lost alliances that ended when their research agenda was complete. Thus, year 4 provided a useful time to describe how the 
alliances evolved and what the mature REL research alliances for the 2012-17 contract cycle looked like. The REL Southwest 
contract started in November 2012, 11 months after the REL contract cycle for all other RELs, so the information as of January 
2015 is reflective of its research alliances in year 3. 

"Ty addition to the 50 U.S. states, the RELs also serve Washington, DC; the U.S. territories of Puerto Rico, the U.S. Virgin 
Islands, American Samoa, the Commonwealth of the Northern Mariana Islands, and Guam; and the Freely Associated States of 
Palau, the Federated States of Micronesia, and the Republic of the Marshall Islands. 

"2 ‘As is discussed in succeeding text, most large-scale research-practice partnership networks in education focus on specific 
regions (for example, the University of Chicago Consortium on School Research) or are coordinated by one organization (for 
example, the Strategic Education Research Partnership). The 2012-17 REL program introduced a multifaceted network of 
research-practice partnerships operated by 10 different organizations across the United States. 

'8 Individual REL and cross-REL collaborative webinars and papers that document and reflect on the REL research alliance 
experience can be found at http://ies.ed.gov/ncee/edlabs/. 


e Provides information about how the RELs were able to create, engage, and maintain multiple 


partnerships; and 


e Informs future collaborative efforts for researchers and practitioners and for those who wish to 


support research-practice partnerships. 


IES and the RELs documented this new endeavor in a variety of ways. The authors of this report relied 


on: 


e Information that RELs provided to IES in fulfillment of contractual requirements; 
e Publicly available material accessed from each REL’s website; and 


e Information compiled through (a) crossREL participatory activities, such as biannual REL 
directors meetings and REL working group meetings, and (b) direct communication with REL 


staff. 


For a more detailed description of each source of information, see table 1. 


Table 1. Sources of information used in this report 


sYolUlce-Meymlalcolanars1é(elal ‘Source description 

REL annual plans Annual reporting of progress assessments of the work, goals, 
challenges, lessons learned, and membership changes for each REL 
alliance was mandatory. 

Research alliance annual updates Annual reporting of research priorities and background information 
for each alliance and of changes to alliance names or operating 
status was mandatory. 

Individual REL websites A section of each REL’s website described the research alliances. 


Semi-annual meetings of REL directors |REL directors, deputy directors, and, in some cases, research 
alliance task leaders attended biannual REL directors meetings that 
included discussion of common research alliance experiences 
across the 10 RELs. The authors of this report participated in nearly 
all meetings and, in some cases, led discussions regarding research 


alliances. 
Minutes and documents from meetings |Cross-REL working groups provided an opportunity for REL staff 
of cross-REL working groups working on similar issues to collaborate to ensure strategic 


deployment of resources across high-priority areas. This report 
incorporates information from meeting notes and summary 
documents from cross-REL working groups. 

Feedback and examples provided by REL |This report was reviewed by REL staff, who provided feedback, 
staff suggestions, details on the membership of their alliances, and the 
examples and vignettes discussed throughout this report. 


Chapter 2, “REL research alliances in the broader context of research-practice partnerships,” focuses on 
how the REL research alliances are similar to and different from other existing collaborative models. 
Chapter 3, “Characteristics of REL research alliances,” describes the evolution of REL research alliances 
over the course of the 2012-17 contract cycle, presents a snapshot depiction of REL research alliances as 
of 2015, and discusses who participated in alliances. Chapter 4, “Challenges and solutions in forming and 
developing REL research alliances,” summarizes RELs’ reports about their experiences in forming and 


developing alliances, including the challenges that REL research alliances faced and the strategies that 
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REL staff and alliance members used to build effective and productive teams, develop and refine research 
agendas, and carry out REL research and technical assistance projects. The conclusion discusses the 


implications of these experiences for future research-practice partnership efforts. 


Chapter 2. REL research alliances in the broader context of research-practice partnerships 


Research-practice partnerships have been in existence for decades (Denis & Lomas, 2003). The healthcare 
industry pioneered the development and use of these partnerships and has subsequently produced the 
most reflective work on them. Researchers have studied extensively the experiences of participants in 
research-practice partnerships in fields such as healthcare, mental health, and social services (Carr & 
Buchanan, 2011; Grella, Hser, Teruya, & Evans, 2005; Ovretveit et al., 2014; Pinto, Wall, & Spector, 
2014; Rawson & Branch, 2002; Riemer, Douglas Kelley, Casey, & Taylor Haynes, 2012). The lessons 
learned from this work have served as a foundation for the initiation of research-practice partnerships in 
other industries, including the educational sector. As the number of education-related research-practice 


partnerships has increased in recent years, so too has interest in understanding how they operate.'* 


Categorizing traditional conceptions of research-practice partnerships in education 


The literature on research-practice partnerships in education largely focuses on those that occur at the 
school or district level or within a specific location (such as a county or city). Because REL research 
alliances vary in geographic spread, often covering multiple communities within and across states in each 
REL’s jurisdiction, the research-practice partnership literature does not directly address the unique 
features of REL research alliances and the challenges that they face. However, this literature is instructive 
because it documents how research-practice partnerships, even those focused on a narrow geographic area, 
can take different forms but often face common challenges. For example, Coburn et al. (2013) identified 
three broad types of educational research-practice partnerships, which differ in important ways structurally 
and operationally but face common strengths and challenges relevant to understanding the REL research 
alliance experience: research alliances in districts, design-based research partnerships, and networked 


improvement communities. 


e Research alliances in districts include partnerships between a district and an independent research 
organization, in which researchers and practitioners have distinct roles. Alliance members, 
consisting of researchers and practitioners, negotiate relevant research questions. Researchers from 
one or more institutions conduct the research in collaboration with practitioner stakeholders who 
provide feedback and review aspects of the research design and instrumentation. Practitioner 
stakeholders may come from one or more school districts or across sectors, such as education, 
youth development, and health and human services, as in the following examples: Research 
Alliance for New York City Schools,’’ University of Chicago Consortium on Chicago School 


Notably, between 2012 and 2014, the William T. Grant Foundation collaborated with the Forum for Youth Investment to 
assemble a learning community for improving research-practice partnerships in education. This effort brought together 
researchers and practitioners who were involved in partnerships to discuss lessons learned and to create a resource for educators 
and researchers who are interested in either embarking on collaborative efforts or strengthening their current efforts. As part of 
this project, the William T. Grant Foundation supported the development of white papers, briefs, and case studies; compiled 
lists of key resources; and created a comprehensive online summary on this topic (see http://rpp.wtgrantfoundation.org/). 


15 . 
See http://steinhardt.nyu.edu/research_alliance/. 


Research (Roderick, Easton, & Bender Sebring, 2009),'° and the Baltimore Education Research 


Consortium.” 


e Design-based research partnerships are based on an engineering model in which researchers and 
practitioners work together to design, develop, and test new instructional or curricular approaches 
in real-world contexts. One example is the Strategic Education Research Partnership,'® an 
organization that works with multiple school districts across the country and with university 
researchers to collaboratively address critical problems of practice. Another example is the 
Vanderbilt MIST Project,'? which worked with school districts to test, revise, and elaborate a theory 


of action for district-wide instructional improvement in mathematics. 


e Networked improvement communities (NICs) bring together educators and researchers in 
multiple settings (for example, multiple districts, schools, or universities) to foster continuous 
refinement and improvement. Similar to design-based partnerships, these networks collaborate 
around shared problems of practice, and they develop, test, and refine initiatives. As opposed to 
design-based research partnerships, which tailor research efforts to particular settings, NICs involve 
similar research efforts across multiple settings—for example, a variety of school districts serving 
varied populations—and thus they can examine not only whether something works, but also for 
whom and under what conditions. For example, the Building a Teaching Effectiveness Network””— 
a partnership of the American Federation of Teachers, the Institute for Healthcare Improvement, 
and the Carnegie Foundation for the Advancement of Teaching—works with individual school 
districts around the issue of improving the effectiveness of new teachers in diverse urban school 
settings. In this case, schools and districts within the NIC engage in multiple rapid-cycle, smaller- 
scale studies, similar to design-based research described above, and then continuously share results 
within the network and make refinements based on what they, as well as their partners, are 


learning. 


These three types of research-practice partnerships share several common elements. The next section will 
compare the common elements of these traditional research-practice partnerships with the experiences of 


REL research alliances. 


How REL research alliances compare to traditionally defined research-practice partnerships 


REL research alliances share many characteristics of traditional education-focused research-practice 
partnerships. In collaboration with REL researchers, alliance members identify policy and practice 
concerns that can be addressed by research, develop plans to collect or access data related to those issues, 
and analyze data to develop and test strategies for educational improvement. RELs use this process to 


work in partnership with alliance members to carry out and disseminate rigorous research studies and 


16 : 
See http://ccsr.uchicago.edu/. 


"" See http://baltimore-berc.org/. 


18 Boas 
See http://serpinstitute.org/. 


1 ; ' 
” See http://peabody.vanderbilt.edu/departments/tl/teaching_and_learning_research/mist/. 


20 we ; . 
See http://www.carnegiefoundation.org/in-action/bten/. 
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technical assistance projects that address the needs of the members. While REL research alliances vary in 
membership and structure, they all include preK through secondary education stakeholders and can also 
include people from colleges and universities, professional associations, advocacy groups, nonprofit 
organizations, foundations, and other entities that are interested in the focus of the alliance. As previously 
discussed, Coburn et al. (2013) provided six common characteristics of education-related research-practice 


partnerships. While REL research alliances incorporate aspects of these common characteristics, they also 


differ in important ways (see table 2). 


Table 2. REL research alliances share many characteristics with other education-related research-practice 
partnerships but differ in key ways 


Common characteristics 


of education-related 


research-practice 
partnerships (Coburn et 
r=) BAO BC) 


Are place-based (often 
within a specific school 
district or community) 


How REL research alliances are 


Similar 
Eight alliances (10 percent of the 79 


alliances) focused on individual districts 
within a state. 


Different 


Seventy-one alliances (90 percent of the 79 
alliances) spanned multiple districts within a 
state (35 alliances) or across states (36 
alliances). 


Make long-term, 
sometimes open-ended 
commitments 


REL research alliance members were 
committed to a multiyear process. 


The REL contract covered a finite, 5-year time 
span. This structure had implications for the kind 
of work that could be conducted during each 
contract year, as alliances experienced a start-up 
phase at the beginning and a wind-down phase at 
the end of the five years. 


Focus on problems of 
practice 


REL research alliances focused on practice- 
related issues. The RELs conducted ongoing 
needs assessments and worked with REL 
alliance members to ensure that work was 
responsive to the practical needs of 
stakeholders. 


In addition to conducting research, technical 
assistance, and dissemination projects to 
address members’ needs, REL research alliances 
also developed activities to support capacity- 
building for members to use data and research- 
based practices to address other emerging 
needs. 


Use intentional 
strategies to foster 
partnership 


REL staff intentionally created opportunities 
for alliance members to meet on a regular 
basis to support the co-creation of a shared 
agenda, plan and implement research 
studies, and review and interpret study 
findings. Alliance members were 
encouraged to take on leadership roles. 


REL research alliances that spanned broad 
geographical areas faced unique constraints on 
being able to hold in-person meetings and 
therefore often relied on virtual meeting formats. 


Produce original 
analyses 


REL research staff developed and 
implemented original research studies 
prioritized by and in partnership with the 
research alliance. 


In addition to conducting original analyses, REL 
research alliances also placed a strong emphasis 
on capacity building, which includes developing 
and providing technical assistance and producing 
tools for alliance members that aim to increase 
practitioners’ understanding of research and 
data. 


Are committed to 
mutualism 


REL researchers and alliance members 
worked together to set priorities and to 
develop and refine a research agenda. 


While REL researchers and alliance members 
worked together to set priorities, develop 
research plans, and carry out projects, all 
research plans and dissemination materials also 
required approval by the funder (IES). 


Note: Coburn et al. (2013, p. 3) define mutualism as “sustained interaction that benefits both researchers and practitioners.” 
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Place-based. Notably, most traditional research-practice partnerships in education are place-based in that 
they bring together diverse stakeholders within a particular location, usually a school district or a set of 
school districts within a specific geographic region (Coburn et al., 2013). Only 8 of the 79 REL research 
alliances operating as of January 2015 were place-based; in other words, the alliances operated in a single 
district, specific districts, or regions within a state. For example, the Silicon Valley Research Alliance 
included 11 school districts in Silicon Valley, California. 


Long-term. The 5-year nature of the REL contracts offers opportunities for research alliances to maintain 
support for multiple years, like other research-practice partnership efforts. Typical research-practice 
partnerships are long-term, open-ended relationships, but their specific projects may have a prescribed 
timeframe and funding opportunities that evolve over time. Unlike many traditional research-practice 
partnerships, REL research alliances were developed with a finite, 5-year time span. In some cases, REL 
research alliances—particularly those that already existed in another form prior to joining the alliance—may 
continue after the end of the REL contract cycle. However, the finite time span of the REL contract had 
implications for the kind of work that could be conducted during each contract year, as alliances 


experienced a start-up phase at the beginning and a wind-down phase at the end of the five years. 


Focus on problems of practice. Like traditional research-practice partnerships, REL research alliances 
were designed to focus on practice-related issues; they were also designed to help practitioners understand 
and use data to inform decisionmaking. RELs were required to conduct ongoing needs-sensing to ensure 
that their work responded to the needs of key stakeholders in their regions. Therefore, RELs were charged 
with: 


e Working in partnership with research alliance members to develop an agenda of complementary 


research, technical assistance, and dissemination projects; 


e Building capacity among alliance members (as well as other practitioners in the regions that they 


serve) to use data and research to inform decisionmaking; and 


e Placing a strong emphasis on supporting capacity-building for alliance members to use data and 
research-based practices to address emerging needs, which is not universally a focus of traditional 


research-practice partnership efforts. 


Use intentional strategies to foster partnership. RELs, like other research-practice partnerships, engaged 
in a variety of strategies to create an infrastructure to support a co-created agenda. Strategies included 
setting up mechanisms for REL staff and alliance members to engage in regular and ongoing agenda 
development, intentional encouragement of alliance members taking on leadership roles, and bringing 
together stakeholders to review and interpret research results. To support these efforts, RELs held regular 
meetings and took part in outreach efforts to maintain participant engagement and ownership in the 
decisionmaking process (which is discussed in more detail in chapter 4). Unlike more traditional place- 
based partnerships, REL research alliances, particularly those spanning broad geographical areas, faced 
constraints (for example, resource limitations and federal restrictions on RELs paying for alliance 


members’ travel) on the methods that they used to hold in-person meetings for alliance members. For this 
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reason, RELs often used communication strategies—such as virtual meetings and webinar formats—to 


facilitate long-distance communication and collaboration. 


Produce original analyses. Traditional research-practice partnerships focus on implementing research 
studies that address questions that the members want answered. RELs, too, engaged in this work by 
developing and implementing original research studies in partnership with the research alliance. For 
example, RELs conducted original research, such as randomized controlled trials or quasi-experimental 
design studies that were deemed high priority by the research alliances. Then, RELs disseminated rigorous 
education research results and research-based practices through in-person events, webinars, newsletters, 


social media, and videos. 


Committed to mutualism. As with traditional research-practice partnerships, a main focus of REL 
research alliances was to foster a collaborative spirit in which alliance members work together and with 
REL researchers to develop, refine, take action on, and stay engaged with a mutually agreed-upon research 
agenda. As the REL and alliance members collaborated to develop this agenda, they also needed to be 
mindful that all research plans and dissemination materials required approval by the funder (IES). In 
some cases, when initially proposed plans were not approved, the REL and alliance members would 
collaborate to refine their plans with a continued focus of making sure that any revisions or new ideas 


continued to be mutually agreed upon. 


This chapter has shown how REL research alliances share many similar characteristics to prior research- 
practice partnerships in education but differ in important ways. Notably, REL research alliances expand 
beyond traditionally place-based efforts, often covering broad geographic areas. The 2017-17 REL 
contract cycle placed a strong emphasis not only on producing original analyses, but also on developing 
tools and providing technical assistance opportunities to support state and local agencies’ capacities to use 
data and conduct their own analyses. While REL research alliances operated as multiyear endeavors, 
similar to traditional research-practice partnerships, they were constrained by the 5-year contract period 
and were subject to additional challenges not typically experienced by other partnerships, for example, 
constraints on the opportunities to meet in person. Importantly, given the scale of the program, REL 
research alliances operating during the 2012-17 REL contract cycle took on a variety of forms. Chapter 3, 
“Characteristics of REL research alliances,” discusses the evolution of the alliances and provides a general 
snapshot of them, which helps to set the stage for later discussions of the challenges that RELs faced and 


the lessons that they learned in implementing the alliances. 
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Chapter 3. Characteristics of REL research alliances 


At the beginning of the 2012-17 REL contract cycle, each REL proposed to work with specific research 
alliances, most of which were newly formed by the REL and its stakeholders. Each REL had broad 
discretion in determining the number, structure, size, and topic area focus of each alliance. Once an 


alliance was established, each REL, often in consultation with alliance members, could decide: 


e Which in-person and electronic strategies it would use to engage alliance members; 
e Whether it would expand or contract the size of the membership; and 


e Whether the alliance would shift focus, merge with another alliance, or conclude because alliance 


research agendas were completed or needs were evolving in the region. 


The REL research alliance experience provides a unique opportunity to explore issues around the start-up 
of new partnerships and learn from the processes of development and evolution of REL research alliances 
over time. This chapter describes the REL alliances as they evolved from year 1 of the REL contract cycle. 

It also provides a detailed snapshot of research alliances as they were configured in spring 2015 (four years 
into the REL contract cycle), a good point at which to examine the fully matured REL research alliance 


experience. 


Evolution of REL research alliances from year 1 to year 4 of the 2012-17 REL contract cycle 


In year 1 of the 2012-17 REL contract cycle, the 10 RELs proposed to work with 73 alliances—that is, 
between 4 and 10 alliances per REL. Fifty-five of the alliances were either newly created or were in the 
“developing” phase, which IES defined as a partnership that is planned and forming but did not have 
plans to initiate formal activities in year 1. As of January 2015, 79 REL research alliances were in 


existence, 70 of which (89 percent) were newly created during the REL contract. 


Between year | and year 4, 15 REL research alliances that were active in year 1 were either no longer in 
existence or no longer in partnership with the respective REL, and 21 new REL research alliances were 
established in partnership with a REL.”! Eleven of the 15 REL research alliances that were no longer active 
came from two RELs that had made significant restructuring changes to their alliance configurations (see 
figure 1). Much of the planned work of these alliances was subsumed into the work of existing or new 
alliances. Among the remaining 4 inactive research alliances across three other RELs, the partnerships 
ended because state-level priorities changed or were too disparate to form a coherent agenda, because they 
accomplished the work from their research agenda, or because the work of the alliance could easily be 


subsumed within a different alliance or other REL support. 


21 To be counted as “no longer in existence,” an alliance must have ended and not transformed into another alliance (in whole 
or part) through changing focus, splitting, or merging. To be counted as “newly established,” a REL research alliance that 
became active after year 1 must not have been based in any way on a year | research alliance. Some RELs may consider a 
research alliance that was split into two separate alliances as the closing of one alliance and the creation of two new alliances. 
However, we use more differentiated definitions for “no longer in existence” and “newly established” to achieve a more 
nuanced depiction of the evolution of research alliances over the REL 2012-17 contract cycle. 
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Figure 1. The number of REL research alliances evolved from 2012 to 2015 
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In addition to closures, many REL research alliances were newly created or changed in some way between 
year 1 and year 4. Appendix B discusses how REL research alliances changed from year 1 to year 4. The 
remainder of this chapter focuses on a snapshot of REL research alliances as they were operating at the 
height of the REL 2012-17 contract cycle. 


The 2015 research alliances at a glance 


As of January 2015, 79 REL research alliances were active across the 10 RELs (see figure 2). Each REL 
maintained between 6 and 10 research alliances, for an average of 8 research alliances supported by each 
REL. Thirty-five of the 79 alliances (44 percent) included members across multiple states, 36 alliances (46 
percent) operated statewide or across multiple districts within a state, and 8 alliances (10 percent) included 


members from one school district or a small subset of districts within a state. 


Figure 2. There were 79 REL research alliances in 2015 


Most REL research alliances On average, each REL supported 8 alliances 
covered multiple districts within 
or across states # of Alliances 
6 35 36 REL Appalachia 6 
3 =f REL Central 7 
§ 20 . REL Mid-Atlantic 8 
= 10 REL Midwest 8 
oO 0 REL Northeast & Islands 8 
9g Year 4 REL research alliances REL Northwest 8 
E m Multistate REL Pacific 10 
2 REL Southeast 8 
@ Single-state, > 1 district REL Southwest 8 
Single-district REL West 8 


Note: For a full list of 2015 research alliances, see appendix A. 
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Each REL research alliance focused, at least in part, on one or more of the topic areas depicted in figure 3 
and discussed in the following paragraphs. In the cases of overlap in topic area, the figure categorizes each 


REL research alliance into the topic area that best defines it.” 


Figure 3. REL research alliances focused on specific topic areas 
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Increasing high school completion and college readiness, access, and completion (25 alliances). About 
onetthird of all REL research alliances focused on issues related to increasing high school completion and 
college readiness, access, and completion. Nearly all (8 out of 10) RELs had a research alliance in this 
topic area. Four of the 25 alliances in this topic area focused primarily on postsecondary education, while 
the others focused primarily on college readiness for school-age youths. In particular, 10 alliances in this 
topic area targeted issues related to dropout prevention or to developing and using early warning systems 
to identify students who are at risk of dropping out of high school. In most cases, these alliances explicitly 
talked about increasing college readiness, preparing for transition to postsecondary, and dual enrollment. 
Because pressing issues and research priorities cross geographic boundaries, it is notable that 10 alliances 


(40 percent) in this topic area included members from multiple states. 


Identifying and retaining effective teachers and principals (21 alliances). Each of the RELs maintained at 
least one alliance in the topic area of identifying and retaining effective teachers and principals. The 21 
REL research alliances in this topic area spanned a number of subtopics, including professional learning, 
teacher preparation, disciplinary policies, formative assessment, virtual education, and educational issues 
related to leadership. More than half (11) of the REL research alliances in this area were created after year 


1 of the REL contract cycle. Nine of the 21 alliances (43 percent) included members across multiple states. 


oat many cases, REL research alliances supported work across multiple topic areas. For example, alliances that focused on 
improving low-achieving schools also may have developed part of their research agenda around teacher effectiveness or special 
populations. For this discussion, we selected the topic area that best reflects the main focus of each alliance. Where a REL 
research alliance title did not fit easily into one category, we reviewed the goals and the nature of the work being conducted by 
the alliance to determine the most suitable topic area. 
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Improving low-achieving schools (21 alliances). All RELs had a least one partnership focused on 
improving low-achieving schools. These alliances covered a wide variety of subtopics, including charter 
schools, school climate, statewide support services, family engagement, and using longitudinal data to 
identify low- or high-achieving schools. Five (24 percent) were created after year 1 of REL work. Six 


alliances (29 percent) included members from multiple states. 


Adopting and implementing rigorous academic standards and assessments (6 alliances). Six REL 
research alliances focused on academic standards and assessments and on issues related to literacy, 
mathematics, and science instruction. Other alliances (such as those that were focused on improving low- 
achieving schools) also touched on this topic in their research and technical assistance projects. Five out of 


six alliances in this topic area included alliance members across multiple states. 


Early childhood education (6 alliances). Five RELs had research alliances that focused on early childhood 
education; one of those RELs had two separate state-specific alliances on this topic. Three research 


alliances had multistate members, and three research alliances included members from one state. 


Some RELs specified populations of interest—such as rural residents, American Indian students, or 
English learner students—to be served by each research alliance. Many REL research alliances covered 
issues related to these populations—either as part of a broader alliance or as a focus of smaller working 
groups. Four alliances focused specifically on rural education; two alliances focused on education of 


American Indian students; and four alliances focused on English learner students. 


Who participated in REL research alliances? 


In January 2015, alliances had 3,416 members, more than half of whom played an active role in the 
planning and implementation of one or more REL activities, such as applied research studies, technical 
assistance-related events, or dissemination of high-quality research.”’ In most cases, REL research alliances 
included a core set of members who took part in regular planning meetings. Often, RELs expanded 
membership beyond this core group for dissemination and participation in planned technical assistance 
activities. RELs characterized these differentiated roles in a variety of ways, but in general, alliance 


members took on one or more of the following roles: 


e Planning team members (“core” members); 


e Advisory group members, including stakeholder advisory groups, who share contextual issues and 


current local needs with the research team; 


e Active members (those who engaged in REL projects or research alliance working groups), which 


also includes core members and advisory group members; and 


°? These counts may be slight overestimates because of alliance members who participated in more than one alliance. RELs 
report total enrollment by REL alliance, so it is not possible to calculate the extent of duplication on the basis of the 
information available. One REL noted that 58 members took part in all alliances, and this information has been factored into 
the estimated total membership count. 
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e Followers, who are on communication and dissemination lists. 


Across the 77 REL research alliances that reported membership information,” individual alliance 
participation in year 4 of the REL contract cycle ranged from 4 members (in the West Virginia Workforce 
Readiness Alliance in partnership with REL Appalachia) to more than 250 (in the Educator Effectiveness 
Research Alliance in partnership with REL Central), with a median of 25 members per REL research 
alliance. This wide range may be due to different definitions that RELs used for “members” and 


“followers.” 


Average “active” membership of REL research alliances, which is likely more consistently defined across 
RELs, ranged from 4 to 100 members: 43 alliances (56 percent) had 15 or fewer active members, 26 
alliances (34 percent) had 16-50 active members, and 8 alliances (10 percent) had more than 50 active 


members (see figure 4). 


Figure 4. Most research alliances had 15 or fewer "active" members 


More than 50 
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16-50 members 15 or fewer 
34% members 
56% 


Note: All data are based on 77 alliances. RELs did not provide complete membership information for two research alliances that ended early in 2015. 


Total and active membership varied across RELs, but most of the variation in membership counts 
occurred within RELs, possibly because of the content and focus of the alliances (see figure 5). In all but 
one topic area (adopting and implementing rigorous academic standards and assessments), alliances were 
more likely to have 15 or fewer active members as opposed to having 16-50 active members or more than 
50 members. For example, two-thirds (67 percent) of alliances focusing on identifying and retaining 
effective teachers and principals had 15 or fewer active members. Alliances across all topic areas were least 
likely to have more than 50 active members. None of the alliances that focused on adopting and 
implementing rigorous academic standards and assessments or early childhood education had more than 


50 active members. 


*4 RELs did not provide complete membership information for two research alliances that ended early in 2015. 
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Figure 5. 2015 REL research alliance membership at a glance 
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Note: All data are based on 77 alliances. RELs did not provide complete membership information for two research alliances that ended early in 2015. 


REL research alliance members included state education agency (SEA) staff, local education agency (LEA) 
staff (including school leaders), and university faculty and researchers. Members characterized as “other” 
include members of regional cooperatives or services centers, other federally funded technical assistance 
centers, staff from private or nonprofit organizations, and professional organizations or consultants. On 
average, 37 percent of members came from LEAs, 32 percent were SEA staff, and the remainder were 


either university based (13 percent) or “other” (18 percent) (figure 6). 


Figure 6. REL research alliance members were predominantly district- and state-level staff members 
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Note: All data are based on 77 alliances. RELs did not provide complete membership information for two research alliances that ended early in 2015. 


Membership composition of research alliances varied mostly within RELs, which suggests that every REL 
supported a variety of stakeholders in different configurations within their research alliances. In particular, 
membership composition depended largely on an alliance’s focus and goals (topic areas), as demonstrated 


by figure 7. For example, across all RELs, LEAs made up 60 percent of the membership in alliances that 
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focused on improving low-achieving schools, compared with 22 percent of the membership in alliances 
that focused on early childhood education. Approximately half of all alliances (38 out of 77) included at 
least one member each from an LEA, SEA, and higher education organization. In addition, 

e Sixty alliances (78 percent) included at least one LEA member; 

e Sixty alliances (78 percent) included at least one SEA member; and 


e Fifty-three alliances (69 percent) included at least one representative from higher education. 


Figure 7. Membership composition varied by topic area 
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Note: All data are based on 77 alliances. RELs did not provide complete membership information for two research alliances that ended early in 2015. 


This “snapshot” has shown the complexity and diversity of the expansive REL research alliance experience. 
The 10 RELs worked with a wide array of research alliances, covering multiple topics across a variety of 
settings and including a wide range of members. Most research alliances were newly created at the 
beginning of the REL contract cycle or commenced over the course of the first few years. REL research 
alliances were not rigidly defined or stagnant entities. These characteristics afforded flexibility and 
evolution as needs emerged or changed, but, as is discussed further in chapter 4, RELs found that this 
flexibility presented some challenges because the responses to changing needs across different research 
alliances could not be addressed through one-size-fits-all strategies. Many newly formed and existing 
research alliances were transformed either in small ways (such as changing their names) or in more 
substantive ways (such as broadening or contracting geographic coverage or combining or splitting 


alliances). 


The work of forming, sustaining, and supporting multiple research alliances that comprised varied topic 
areas, geographic areas, and structures was an undertaking in which the 2012-17 RELs were breaking new 
ground. In this endeavor, REL staff faced and adapted to a variety of challenges during the course of their 
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work. The remainder of this report delves more deeply into the experiences of the RELs and explores the 


implications for other research-practice partnerships. 
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Chapter 4. Challenges and solutions in forming and developing REL research alliances 


Over the first four years of the 2012-17 contract cycle, RELs reported that they faced a variety of 
challenges and used a number of strategies to meet these challenges as they supported the formation of 79 
alliance partnerships, engaged in activities to develop and maintain a solid membership base and 
structure, and worked with research alliance members to develop and refine their research agendas. This 
chapter examines the broad categories of challenges that RELs faced, discusses strategies that the RELs 
reported using to address these challenges (summarized in table 3), and cites specific examples from across 
the REL research alliance effort. 


REL research alliances reported facing both common and unique challenges 


The REL research alliance experience provides a unique opportunity to explore issues concerning the 
start-up of new partnerships and the processes of development and evolution over time. Ten different 
RELs supported the development of research alliances, and, in year 4 of the contract cycle, each of these 
RELs supported between 6 and 10 alliances that varied in origin, size, geographical spread, topic area, and 
diversity in membership. The RELs encountered a wide variety of challenges and enacted a number of 
strategies to meet these challenges. Some of these challenges were similar across alliances, while others 


were specific to a particular alliance.” 


RELs reported commonly facing the following categories of challenges when developing or supporting 


alliances: 


e Building trust, engagement, and a sense of partnership; 
e Fostering communication; 
e Managing time constraints; 


e Working with research alliance members who have different priorities, interests, and expectations; 


and 
e Building capacity. 


Table 3 below provides specific issues reported by RELs related to each of these broad challenges, as well 
as approaches taken by RELs to address each broad challenge. The RELs found that many of these 
approaches addressed more than one challenge. Further, this chapter captures the basic descriptions of 
approaches reported by RELs to address the respective challenges but does not provide detailed depictions 
of how these approaches were implemented. 


All 10 RELs provided IES with yearly written reflections that addressed challenges and lessons learned in the form of updated 
annual planning documents and annual progress assessments. RELs also provided oral reflections to IES during biannual REL 
directors meetings. This chapter contains information excerpted directly from REL documents or statements provided by REL 
directors and other REL leadership team members. 
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Table 3. REL research alliances reported facing a number of challenges and responding to these challenges 


in specific ways 


Category of 
challenges 


RELs reported facing these 


specific challenges 


REL research alliances responded to these challenges by 


Building trust, |e Establishment of new e Taking advantage of previously developed leadership 
engagement, relationships among alliance structures when possible 
anda sense of | members e Using REL staff and alliance members who could 
ownership e Development of relationships navigate the researcher-practitioner divide 
between alliance members —e_ Encouraging alliance members to take on leadership 
and REL staff roles 
° Cultivation of active __ © Fostering a core team of engaged alliance members, 
participation of membership often with differentiated roles 
Fostering e Competition among schedules je Facilitating early and ongoing in-person and virtual 


communication 


Limited resources for holding 
face-to-face meetings 


Technology barriers 


e Lack of engagement of staff 


from alliance member 
organizations or agencies 
Reduced involvement of 
members due to geographic 
distance 


meetings by using creative and flexible methods 
Combining formal, ongoing communication methods 
with less formal, as-needed discussions 

Taking part in regular communication and dissemination 
events with nonalliance members and key stakeholders 


Designating specific staff to facilitate the exchange of 
information 


Managing time 
constraints 


Competition of obligations 
among alliance members and 
other key stakeholders 
Choice of projects that would 
provide timely feedback to 
inform policy and practice 
The finite 5-year nature of the 
REL contract 


Factoring in members’ time restrictions and competing 
obligations when considering membership roles 
Continuing to recruit and motivate new members to 
replace those with insufficient time 

Finding creative ways to support members when there 
were time lags in dissemination of findings 

Developing and (when necessary) revising realistic 
timelines that fall within the contract period and meet 
the needs of stakeholders 


Working with 
research 
alliance 
members who 
have different 
priorities, 
interests, and 
expectations 


Differences in disciplinary 
training of researchers and 
practitioners 

Differences in priorities among 
practitioners due to their 
respective positions in their 
communities 

Varied local needs, priorities, 
and interests across different 
communities within a research 
alliance 

Changes in policy contexts 
over time 

Ongoing issues related to 
power dynamics 


Working to find common ground and developing a core 
agenda around common interests 

Participating in ongoing reflective work and regularly 
recommitting to the evolving research agenda 
Identifying power dynamics and working to develop a 
positive group dynamic 

Being flexible, particularly when there were changes in 
state and local leadership or legislation 

Subdividing an alliance on the basis of geographic areas 


e Developing smaller, more cohesive workgroups within 


an alliance 


e Supporting local or regional needs while maintaining a 


core research agenda 


Building 
Capacity 


Variation in capacity of alliance 
members to understand 
research methods and frame 
research agendas 

Tension between building 
capacity and advancing the 
research agenda 


Developing toolkits to serve a broad base of alliance 
members and their colleagues 


Providing targeted technical assistance to meet the 
individual capacity-building needs of member 
organizations 


Using locally based staff to help build capacity 
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Challenge 1: Building trust, engagement, and a sense of ownership 


Trust in research-practice partnerships requires early and ongoing collaboration and a feeling of mutual 
ownership in decisionmaking (Coburn et al., 2013; Connolly, Plank, © Rone, 2012; Gooden, Graham, & 
Martin, 2014; Tseng, 2013). As one REL director reported during a cross-REL meeting, “Relationship and 


trust need to be developed before members can collaborate and do the ‘research work’.” 


How did RELs report building trust, engagement, and a sense of ownership as a challenge? 


Trust and relationship-building are interconnected with all of the challenges that RELs reported facing. 
However, the RELs reported three specific challenges with regard to building trust: 


e Establishment of new relationships among alliance members. In many cases, particularly with 
newly created REL research alliances, members had never met before the formation of the alliance. 
Even in cases where members knew each other, they may not have worked together often, 


particularly in the capacity of developing a shared research agenda. 


e Development of relationships between alliance members and REL staff. Even in cases where 
alliance members already knew one another well, particularly for alliances that existed before the 
REL contract began, RELs reported that alliance members needed to develop and cultivate a new 


working relationship with REL research and support staff. 


e Cultivation of active participation of membership, including supporting the development of 
leadership roles. When members are not engaged and taking on key roles, then they feel less 
ownership in the process. Regardless of how alliances were formed, RELs reported needing to find 


ways to support the development of leadership and mutual ownership among alliance members. 


How did RELs and REL research alliances report responding to these trust, engagement, and sense of ownership 
challenges? 


Because trust and a sense of ownership act together as a cornerstone to all of the work of the alliance, 
RELs reported undertaking numerous strategies to respond to this challenge. The following four strategies 
were identified by the RELs to help build trust and to foster a sense of mutual ownership in 
decisionmaking: 


e Taking advantage of previously developed Tihen euuaL ihn Se Sols ERLE 


leadership structures when possible. RELs partnerships that the REL did not create], it doesn’t 


that worked with existing groups indicated make sense (in most cases) to create an additional 
that it is often important to rely on an entity's REL leadership structure on top of or alongside 


the existing leadership. The REL works with the 
existing governance structure. You don’t want 
separate governance structures.” 


—REL director 


than creating a new leadership structure 
solely for alliance work and planning. 
Specifically, individuals and organizations 
that have worked together in the past have 


likely already established trust and an effective dynamic of working with each other. In this context, 
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REL researchers and staff had to work to build trust as outsiders who were new partners with 


alliance members. 


e Using REL staff and alliance members who could navigate the researcher-practitioner divide. All 
RELs designated staff to work as liaisons with REL research alliance members. Although RELs used 
a variety of staffing arrangements, each REL designated staff members with research, practice, 
specific content knowledge, or some combination of these areas of expertise as lead contacts for 
each alliance. These lead contacts helped build the research alliance partnerships and worked to 
keep alliance members engaged and on the same page. They attempted to navigate the researcher- 
practitioner divide by establishing one-on-one relationships with members, participating in and 
coordinating frequent communication, and identifying specific members to serve as champions for 
the alliance or serve in lead or advisory roles. RELs reported that the most effective research 
alliance liaisons were those who could easily bridge the worlds of researchers and practitioners and 
could speak the “dual language” of research and practice.”° One way RELs reported developing this 
skill was through creating opportunities for liaisons within an individual REL to meet regularly 


with each other to share best practices in working with practitioners. 


e Encouraging alliance members to take on leadership roles. REL research alliance members— 
particularly those who volunteered or were selected by the REL to take on leadership roles—played 
critical parts in building trust. RELs reported that core alliance members set the tone of meetings 
and discussions and took actions that fostered joint cooperation and buy-in: they aided in member 
recruitment, guided the rollout of new projects, and acted as “dissemination ambassadors.” Again, 
RELs noted that research alliance members who could also speak “languages” of both researchers 
and practitioners helped to foster communication in ways that reduced distrust and enabled all 
participants to have a say in the substance of the alliance’s work. Having a say, in turn, supported a 
stronger sense of ownership in the work of the REL research alliance. In addition to volunteering 
or being selected by the RELs, people taking on leadership roles also had to be willing to commit 
the time needed to support implementation of the alliance’s agenda. While the goal was sustained 
participation in leadership activities by alliance members, RELs acknowledged that this was not 
always possible. In some cases, for example, when attendance was irregular, RELs would sometimes 
work with these leaders to explore whether to recruit an alternate representative who could take on 
a more committed role. In these situations, recruitment of a committed leader took precedence 


over whether the original alliance member was retained. 


e Fostering a core team of engaged alliance members, often with differentiated roles. In their 
discussions, RELs agreed that a big part of building trust included bringing together a strong and 
productive team of researchers and practitioners to serve as members in REL research alliances. 
Organizational structures and levels of participation among members varied across REL research 


alliances. While some RELs supported alliances that used a formal approach to the structure of 
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Ideally, these alliance liaisons should have some experience or expertise in both education research and practice so that they 
can “speak the language” of both researchers and practitioners, help facilitate their collaboration, and make sure both groups’ 
needs are being met through the research alliance partnership. 
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alliance roles, such as two- or three-tiered systems (including, for example, “core” planning team 
members, “active” members who are engaged in REL projects or working groups, and “followers” 
who are on communication or dissemination lists), others saw value in less rigid roles for alliance 


members to play. 


Fostering a core team of engaged alliance members: an example from REL Central 
REL Central created alliance planning committees to bring together a group of engaged 


alliance members and foster a sense of ownership. Planning committee members met 
regularly to help shape and guide each alliance’s evolving research agenda. Each planning 
committee operated as an advisor to REL Central about the work conducted by the 
respective alliance and structured itself in its own way. For example, the planning committee 
of the Rural Education Research Alliance created subcommittees to guide the development 
of work within each member state; whereas, the Educator Effectiveness Research Alliance 
brought together a member representative from each state into one main committee. 


REL Central reported that this flexibility in alliance planning committees enabled research 
alliances to operate in ways that best addressed each alliance’s specific needs. In the case 
of the Educator Effectiveness Research Alliance, which was created in year 2 through the 
merging of state-specific alliances, the planning committee helped to oversee the work 
across states and shape alliance-wide priorities. Similarly, the Rural Education Research 
Alliance planning committee members provided overarching support and fostered 
engagement among other alliance members through the creation of smaller project teams 
that could focus their time on work more aligned to their respective interests. 


Regardless of the roles that members played, RELs reported that building trust required the establishment 
of a strong and dedicated membership base and the provision of ample opportunities for relationship- 
building—both during initial planning and throughout the lifespan of the research alliance. One way to do 
this was to support opportunities for members to discuss their roles both within and outside of the 
alliance context. For example, one strategy that REL Appalachia used was to allow time during meetings of 
the alliance or core planning teams for members to network by sharing examples of other, non-REL work 


in which they were engaged. 


Challenge 2: Fostering communication 


Building trust requires regular and effective communication. Communication is perhaps one of the 
biggest hurdles to overcome in a research-practice partnership, particularly because it relates to members 
who are geographically spread out and do not have frequent face-to-face contact. Communicating by using 
various methods and channels and by using content that is relevant helps keep members engaged, 
invested, and contributing to the overall success and sustainability of the research-practice partnership 
(Roderick et al., 2009; Smith, Bingman, & Beall, 2006; Vukotich, Cousins, & Stebbins, 2014). Unlike 
many other education-related research-practice partnerships that focus on a specific location, such as a 
school district, REL research alliances varied widely in their geographical coverage, making regular face-to- 
face communication more challenging. By year 4 of the 2012-17 REL contract cycle, more than 40 
percent of REL research alliances had memberships that crossed state lines within a REL’s region. Even 


single-state REL research alliances sometimes covered multiple, diverse school districts and settings. 
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How did RELs report fostering communication as a challenge? 


RELs reported that strong and consistent communication among alliance members generally brings forth 
a sense of personal responsibility and commitment to alliance work. However, RELs reported facing the 


following five communication-related challenges: 


e Competition among schedules. RELs reported that members of research alliances have busy 
schedules and competing demands across their various roles within and outside of the alliance. 


This makes it difficult to schedule meetings that include all alliance members. 


e Limited resources for holding face-to-face meetings. Although REL staff acknowledged that in- 
person meetings were preferred, particularly during the agenda-setting stage, they stated that such 
meetings were not always practical or possible, because of restricted travel expenses”’ and because of 


the schedules and travel limitations of alliance members. 


e Technology barriers. Some members of REL research alliances lived in remote or rural regions 
with limited technological capability or reliability. RELs reported that not only is it difficult for 
these members to convene in person because of their geographic constraints, but it can also be 


difficult to meet via platforms that require Internet access. 


e Lack of engagement of staff from alliance member organizations or agencies. RELs acknowledged 
that stakeholders outside of the alliance do not participate in as regular and consistent 
communication as alliance members and thus face the potential to lose sight of the alliance’s 
progress. RELs noted that communication between research alliance members and nonmember 
stakeholders (such as state and school district leaders) is essential to gaining support for 
implementing REL research and technical assistance projects, as well as to bridging the gap between 
research and practice. These leaders may not be directly involved in developing and engaging in 
alliance activities, but they are critical in providing support for staff members who are taking part in 
these activities and in reducing barriers to successful implementation of research and technical 


assistance projects. 


e Reduced involvement of members due to geographic distance. RELs reported that member 
involvement declined because of a lack of close connection and interaction of members of research 


alliances that covered multiple geographic locations. 


How did RELs report responding to communication challenges? 


To address these challenges, RELs stated that they needed to be creative and flexible in fostering 
communication within research alliances. RELs developed a portfolio of in-person, phone, and virtual 


meetings to keep the lines of communication open. Different RELs reported using different strategies, and 


” As discussed in the 2012-17 Performance Work Statement, “REL funding cannot be used for invitational travel; therefore, 
the contractor shall not provide federal reimbursement for travel or meal payments to recipients of REL services. Alliance 
members will need to pay for their participation in meetings.” See the “Performance Work Statement for Regional Educational 


Laboratories (2012-2017)” available at https://Awww.fbo.gov/utils/view’id=50688bfda3cc31d77469838a79649491. 
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even within a REL, different research alliances often required different communication strategies. In 


general, RELs reported responding to communication challenges by using the following strategies: 


e Facilitating early and ongoing in-person and virtual meetings by using creative and flexible 
methods. RELs discussed the importance of having in-person meetings when possible and took 
advantage of opportunities to bring alliance members together. For example, REL Central 
addressed the challenge of travel payment restrictions by capitalizing on alliance member 
attendance at both national and state conferences and organizing in-person alliance meetings before 
or after the conferences. Depending on specific circumstances, RELs may have facilitated in-person 
meetings during the early alliance planning stages, annually, two to three times per year, quarterly, 
or monthly. RELs adjusted in-person meetings to reflect the needs and logistics of specific alliances. 

In some situations, a smaller subset of alliance “There really is no replacement for 
members (for example, a workgroup within an handshake, face-to-face interactions, and in- 


alliance) met in person more regularly than the larger, PESO" dialogue. 


whole group. RELs also sometimes hired locally based | —REL Year 5 Research Alliance Summary 
consultants or had REL staff conduct site visits to 

alliance members’ work locations to foster more face-to-face interactions with alliance members. In 
other cases, RELs combined in-person meetings with virtual meetings—either by phone or video 
conference—to ensure that all research alliance members were included in the conversations, 
regardless of their ability to attend a meeting in person. For alliances that faced technology barriers, 
RELs explored solutions such as dividing one big alliance into separate and smaller geographically 
focused alliances or offering in-person meetings that were more efficient in terms of time and 


travel. 


e Combining formal, ongoing communication methods with less formal, as-needed discussions. In 
addition to formal meetings, the RELs reported using other formal communication strategies, such 
as regular (monthly or bimonthly) email briefs or newsletters, to keep alliance members informed of 
alliance activity and to foster communication between members. They experimented with creative 
formats in these newsletters, including adding video clips or infographics. In addition, all RELs 
supplemented more formal, scheduled communication with less formal, as-needed emails, phone 
calls, blogs, and members-only Web-based discussions. Although RELs acknowledged the 
importance of ongoing and diverse methods of communications, one REL pointed out that alliance 
leads must engage in a “careful balancing act” of keeping participants engaged but not 


overburdening them with too much outreach. 


e Taking part in regular communication and dissemination events with both alliance and 
nonalliance members and key stakeholders, such as state and district staff. RELs reported taking 


part in the following engagement strategies with both alliance and nonalliance members: 


1. Disseminating early research study findings to a limited set of stakeholders, including state or 


district leaders; 


2. Encouraging alliance members and other stakeholders to participate in REL events (and 


monitoring state and district calendars to minimize potential time conflicts); 
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3. Partnering with alliance members to present information to larger audiences; 
4. Providing toolkits or other resources that alliance members can use to train other practitioners; 


5. Employing on-site staff or local consultants who have the capacity to communicate more 


regularly with local stakeholders; 
6. Preparing videos and infographics to disseminate work; 
7. Forming advisory groups; and 


8. Hosting meetings or events across the region’s alliances. 


Using multiple techniques to keep the communication lines open: an example from REL Pacific 
REL Pacific used a variety of communication strategies aimed at keeping alliance members 


informed and engaged. Regular communications through electronic “infoblasts” were sent to 
alliance members at least twice per month. These communications provided information about past 
and future meetings, included pictures of alliance members, and provided information about the 
work of the alliances. “Events and Resources” infoblasts provided details regarding relevant 
webinars and events. REL Pacific also connected with alliance members informally through emails 
and phone conversations to schedule meetings, respond to questions, and gather feedback. They 
also took advantage of locally based consultants to maintain close connection with alliance 
members. 


RELs discussed the importance of engaging in an intentional “no surprises” communication 
approach. Because nonalliance members are not as informed of the day-to-day operations of the 
alliance work, there is a real threat that they will lose sight of the alliance’s progress and their policy- 
relevant work and be less amenable to encouraging the continued progress of the research alliance’s 
agenda. RELs reported that carefully planned communication strategies, such as the eight listed for 
this strategy, made key stakeholders (particularly district and state leaders) aware of ongoing alliance 


: work with the hope that they would continue to support staff members 
RELs engaged in an 


intentional “no surprises” 
communication approach 


who were taking part in research alliance-related activities and to help 
minimize barriers to successful implementation of research alliance 


projects. 


e Designating specific staff to facilitate the exchange of information. Some RELs reported that they 
found it useful to designate specific staff to focus on communications and act as the facilitators of 
information exchange within an alliance. Designated staff could be members of the REL staff or of 


the research alliance. 


Challenge 3: Managing time constraints | 
“We are not talking just about meeting time 


All REL research alliances faced the shared pressure of when we talk about the time needed for the 
limited time. Time constraints influenced issues related to development of relationships and trust; we 
alliance membership, setting research agendas, scheduling _4f€ talking about calendar time.” 

meetings, and implementing research and technical —REL director 


assistance projects. 
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How did RELs report managing time constraints as a challenge? 


In general, there were three categories of time constraint challenges: 


e Competition of obligations among alliance members and other key stakeholders. Alliance 
members played various roles within and outside of their alliances, which contributed to challenges 


surrounding facilitation of REL research alliance 
“Because of their busy schedules and 


competing priorities, the research alliance 
members often have limited time to engage 
as thoughtful partners.” 


—REL Year 5 Research Alliance Summary 


work. In addition to barriers to communication, 
competing obligations made it difficult to engage 
more generally in the work of the alliance. For 
example, district or state members who were 
experiencing changes in policy or legislation related 
to their jobs may have had to pay full attention to new career-related activities or initiatives, making 
it difficult to also engage in alliance work. In some cases, this led to turnover in REL research 


alliance membership or to reduced participation in alliance activities. 


e Choice of projects that would provide timely feedback to inform policy and practice. REL 
research alliances needed to negotiate not only what projects were of the highest priority, but also 
which ones would be realistic, given time constraints. RELs reported that some projects that could 
have been valuable to an agency or department in the long run were not considered, because the 
research window did not allow enough time to produce results that would inform imminent policy 
or practice decisions.”® REL research alliances therefore had to work carefully to identify research 
priorities and projects that would produce policy- and practice-relevant results that could be 
addressed within a limited timeframe. Also, district schedule constraints included blackout dates, 
such as holidays, state testing, and student summers off. RELs noted that these restrictions, coupled 
with the urgency to provide stakeholders with timely reports and preliminary information, placed a 
time burden on implementing research studies in school-based settings or finding opportunities to 


provide technical assistance for school or district staff. 


e The finite 5-year nature of the REL contract. The 5-year contractual obligations that RELs were 
under provide a finite timeframe for developing and producing research and technical assistance 
work. Therefore, long-term research projects, such as large-scale randomized controlled trials, could 
not be considered because the time constraints of the REL contract required projects to come 
together quickly and carefully to fit within the 5-year window. This challenge became particularly 
prominent later in the contract cycle when new needs arose that required a longer time window to 


address than remained in the contract period. 


*8 These time constraints are not limited to the REL experience. Coburn et al. (2013) discuss how this phenomenon affects 
most research-practice partnerships. They suggest that creating timelines that satisfy research-practice partners can be tricky 
when trying to balance the need to construct high-quality and rigorous research studies with the need to produce time-sensitive 
results for district decisionmakers. 
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How did RELs report responding to time constraint challenges faced by their research alliances? 


RELs reported learning, over time, that they needed to be realistic about availability of members’ time, 
develop reasonable timelines, and focus on projects that were both policy relevant and feasible. 
Specifically, the RELs reported confronting issues related to time constraints by using the following 


strategies: 


e Factoring in members’ time restrictions and competing obligations when considering 
membership roles. RELs reported creating various configurations of alliance membership to 
accommodate members’ time limitations. Those who took on leadership roles needed to be willing 
and able to commit significant time to the REL research alliance; while others who wanted to stay 
connected to the alliance but were time constrained could participate in other ways and take on 


fewer responsibilities within the alliance. 


Finding ways to bridge multiple commitments: an example from REL West 
In some cases, RELs were able to take advantage of alliance members’ multiple roles by 


coordinating activities with other entities in which alliance members were involved. REL West's 
Educator Effectiveness Alliance benefitted from coordination between REL West and the West 
Comprehensive Center (WCC), with which many alliance members were also affiliated. The WCC 
convened regular cross-state events and provided opportunities for face-to-face meetings, which 
REL West supplemented with virtual communication. Each meeting included a research update. 
Research by REL West was shared with all three of the participating states and was used for 
decisions in the state that collaborated directly in a given study. The WCC also incorporated the 


information into their technical assistance to states on implementation of state systems. 


e Continuing to recruit and motivate new members to replace those with insufficient time. RELs 
reported that the alliances often continually recruited new members, not just to expand 
membership, but also to be in a position to replace members who had to leave because of 
competing demands or changes in priorities. In some cases, RELs reported that outgoing alliance 
members helped to recruit their replacements. Because turnover was a common phenomenon, 
RELs also reported the need to quickly engage new members, provide technical assistance when 


necessary, and have clear documentation of the accomplished work to date. 


e Finding creative ways to support members when there were time lags in dissemination of 
findings. RELs noted that, precisely because alliance members were busy, regular contact was 
necessary to keep members engaged. REL research alliances tried various ways to combat time lags 
that threatened to disengage alliance members—particularly with long-term research projects that 
required more time to publish findings. To aid stakeholders who needed information to make 
timely policy and practice decisions, RELs sometimes disseminated existing evidence-based 
resources or provided targeted technical assistance on emerging knowledge on a specific topic. For 
example, REL Southeast’s Improving Literacy Research Alliance implemented a study of the effect 
of Mississippi's K-3 early literacy professional development initiative on teacher outcomes, which 
required a lengthy data-collection process. While that study was still in progress, elementary school 
principals were in need of guidance as they embarked on new summer reading camps. Two research 


alliance members, who were co-authors with REL staff on the research study that was still in process 
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and were members of the Mississippi Department of Education, worked in collaboration with the 
alliance lead to develop and implement regional workshops for principals on using the Summer 
Reading Camp Self-Study Guide, which was previously developed by the REL. 


Developing and (when necessary) revising realistic timelines that fall within the contract period 
and meet the needs of stakeholders. The time constraint of the 5-year contract required members 
to come together rather quickly. Long-term projects were developed early, and in later years, new 
projects were designed to be shorter in duration. In the middle years of the contract, REL research 
alliances often chose to refine research agendas and plan projects that built on prior or ongoing 
work. In year 5 research alliances prepared to wind down their research agenda and focus more on 
dissemination of findings in preparation for the contract’s close, and some discussed possibilities 
for sustainability beyond the 2012-17 REL contract cycle. 


Challenge 4: Working with research alliance members who have different priorities, interests, and expectations 


REL research alliances faced the challenge of managing competing priorities and interests throughout 


their formation, agenda setting, project development, and implementation phases. 


How did RELs report differences in priorities as a challenge? 


The REL research alliances noted that differences in priorities were due to the following factors: 


Differences in disciplinary training of researchers and practitioners. REL research alliances often 
had to identify and negotiate research agendas that met the needs of both their researcher and 
practitioner members. Researchers often had expertise in and preference for more complex 
research methods that required longer timelines or aimed to answer broader questions; while 
practitioners who worked within different institutions and bureaucracies were often interested in 


quicker answers to specific and tailored questions. 


Differences in priorities among practitioners due to their respective positions in their 
communities. Practitioners in the same alliance often had different priorities, depending on 
whether they represented particular localities or broader communities or interests (for example, 
local-level versus state-level education professionals). RELs reported that research alliances had to 


find a balance between the individual interests of each member. 


Varied local needs, priorities, and interests across different communities within a research 
alliance. Because of the diversity across alliance members, RELs reported that they had to ensure 
that alliance members felt buy-in and ownership and sought common ground. Varied needs, 
interests, and priorities often plagued newly created alliances more than previously existing ones 
because members had less experience working together toward a common goal. However, RELs 
reported that even previously existing partnerships were not immune to the challenge of finding 


common ground, given varied local priorities. 


Changes in policy contexts over time. Priorities of REL research alliances were affected over time 
by changes in school, district, and state leadership and by legislation changes. RELs often reflected 
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on the constant ebb and flow of local or state priorities. For example, local priorities sometimes 
shifted because of changes of district or state leaders (such as new superintendents), who set new 
priorities. Changes to leadership and legislation can affect research alliance membership, member 


engagement, research priorities, access to data, and overall relationship and power dynamics. 


e Ongoing issues related to power dynamics. RELs reported that power dynamics came into play 
throughout all phases of the REL research alliance experience (during initial formation, agenda 
setting, and alliance change and evolution). RELs noted that power dynamics could occur in 
relation to the positions that alliance members held outside of the alliance context (for example, 
state versus district representatives) or in relation to the different roles that members took within 


the research alliance (leadership or core members versus less active members). 


How did RELs report responding to differences in priorities among alliance members? 


RELs reported the following ways of dealing with issues around different priorities: 


e Working to find common ground and developing a core agenda around common interests. In 
order to manage competing interests, RELs reported that research alliances strove to find 
commonality among individual research interests and use those common themes to shape the core 
of the alliances’ work. In a few circumstances, when they were not able to bridge these differences 
between priorities and interests of members to identify a common project, RELs decided to end 


certain alliances or break them into multiple, more cohesive research alliances. 


e Participating in ongoing reflective work and regularly recommitting to the evolving research 
agenda. Given that priorities of all stakeholders would naturally change over five years, RELs 
acknowledged that research agendas were always evolving. In some cases, REL research alliances 
chose to narrow their scope of work to address the most pressing needs initially identified by 
alliance members, while in other cases, alliances chose to broaden work over time to address new or 
additional needs. On an annual basis, RELs were required by IES to reflect on the prior year and 
prepare for the upcoming year’s work. As part of this reflection, RELs worked with research 
alliances to develop new projects or revise work plans when necessary. In some cases, RELs reported 
that REL research alliances asked members to commit formally to research priorities and goals on a 


yearly basis to help ensure that everyone was engaged and dedicated to the research agenda. 


e Identifying power dynamics and working to develop a positive group dynamic. RELs reported 
that power dynamics needed to be acknowledged and understood in order to minimize the chance 
that agendas become driven by only a small subset of members. To address this concern, RELs 
suggested that alliances foster open dialogue among diverse alliance members and focus on issues 


pertinent to all participants. 


e Being flexible, particularly when there were changes in state and local leadership or legislation. 
RELs reported that existing alliance members needed to build trust and positive relationships with 
new leaders in order to encourage buy-in. RELs reported actively working to engage new leaders by 
discussing their priorities, bringing them up to speed on REL research alliance work, and providing 


them with examples of how the current or future alliance efforts are addressing the needs of their 
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constituency. Some alliances evolved and adapted to the changes in leadership or legislation by 
restructuring their agendas and projects to suit the priority areas of new leaders. Other alliances 
ended because their work was no longer relevant or applicable or because new leadership had not 
engaged in the alliance enough to be successful. RELs have indicated that maintaining flexibility, 
being responsive to changing needs, and keeping lines of communication open were essential to 


overcoming challenges surrounding leadership and legislation changes. 


e Subdividing an alliance on the basis of geographic areas. In some cases, when RELs determined 
that alliance needs were too disparate, RELs reconfigured alliances to cover smaller geographic 
regions and developed cross-state collaborative opportunities to facilitate the sharing of resources of 


common interest. 


e Developing smaller, more cohesive workgroups within an alliance. In other cases, alliances 
developed smaller workgroups within the broader group to focus on issues that were of particular 


interest to specific communities or members. 


e Supporting local or regional needs while maintaining a core research agenda. RELs reported 
tailoring technical assistance efforts or producing targeted materials to meet the needs of specific 
communities within the alliance while also maintaining a “core” focus on common needs related to 
policy and practice across all communities. For example, REL Appalachia’s Virginia Middle School 
Research Alliance had a broad focus on expanding data use by practitioners to improve teaching 
and learning. The alliance brought together a collaborative working group of researchers, state-level 
staff, university partners, and local district- and school-level staff to catalog current data-collection 
practices, examine what helps and hinders data use, and identify the most appropriate data to track. 
Importantly, the alliance also provided targeted support to school districts. The alliance developed 
district-specific workshops on using data or conducting action research. The alliance also provided 
technical assistance support to districts or the state in analyzing specific data relevant to their 


pressing needs and interests, such as labor market or disciplinary data. 


Challenge 5: Building capacity 


The 2012-17 REL program placed a strong emphasis on building capacity to understand and use data and 
research effectively and productively to inform decisionmaking.” Many REL research alliance projects 
therefore aimed to build this capacity among research alliance members and other practitioners in the 
regions served by the RELs. Researchers and practitioners came from different contexts or perspectives, 
had different understandings of research methods and how to translate research into practice, and had 
different levels of experience with framing research questions and agendas. RELs reported that capacity 
building was particularly complicated because of diversity in understanding of research methods and data 


use among alliance members. 


” This effort to improve capacity addresses an ongoing challenge in the field of research-practice partnerships. For example, 
authors frequently discussed the need for building the capacity of practitioners participating in research-practice partnerships to 
use research findings to influence decisionmaking and practices (Coburn et al., 2013; Nelson et al., 2009; Roderick et al., 2009; 
Tseng, 2013). 
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How did RELs report building capacity as a challenge? 


Many RELs reported the following challenges related to building the capacity of research alliance members 


to understand and use data and research: 


e Variation in capacity of alliance members to understand research methods and frame research 
agendas. RELs reported that it was difficult to assess alliance members’ capacities to use and 
conduct research, but there was clear variation across alliance members. Variation in capacities 
became apparent as alliance members grappled with the challenge of developing and refining 
research questions that they would be able to answer, given the data that they could obtain. In 
some cases, the research capacity of the alliance partners affected whether projects were short or 
long term. One REL reported that an alliance’s members that had strong in-house research capacity 
within their agency or organization were more likely to engage with the REL on longer-term 
projects; whereas, alliances whose members had little capacity in-house were more inclined to focus 
on answering more pressing, short-term questions. Given the variation in capacity and needs of 


different members, the RELs were challenged to customize strategies to build capacity. 


e Tension between building capacity and advancing the research agenda. RELs faced a challenge in 
advancing the research agenda at a steady pace while also building the capacity of alliance members 
to help implement research projects and frame the research agenda. Because some alliance 
members lacked skills in, for example, developing strong and answerable research questions and 
determining appropriate analytic strategies, RELs used technical assistance activities to strengthen 
the skills of alliance members. This naturally led to longer timelines in formulating and carrying 
out the research agenda. However, at the same time, REL research alliances faced pressure to be 


responsive to the research needs of their members and to produce results in a timely manner. 


How did RELs report responding to challenges of building capacity? 


RELs reported engaging in the following efforts to enhance the capacity of alliance members to 


understand and use research and data: 


e Developing toolkits to serve a broad base of alliance members and their colleagues. RELs 
developed technical assistance activities and tools that provided support to alliance members and 
also could be used by alliance members to build capacity among other staff within their respective 
organizations or agencies. For example, some RELs developed toolkits on particular topics and then 
trained alliance members on how to use the toolkits to support the development of knowledge 


among other state and local personnel. 
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e Providing targeted technical assistance to meet the individual capacity-building needs of member 
organizations. The RELs often worked with research alliance members and their respective 
organizations or agencies to develop technical assistance that supported their specific needs and fit 
within the larger goals of the alliance. In addition to building capacity for alliance members and 
their colleagues, RELs reported that these efforts helped build or solidify strategic relationships 
within and outside of alliances, energize key stakeholders around the goals of the alliance, and 


move the work of the research alliance forward. 


Developing technical assistance activities and tools to meet the needs of practitioners: an 
example from REL Appalachia 
REL Appalachia administered a survey to gather information for the Metro Nashville Public Schools 


Data Use Alliance on how teachers were using data and to determine the supports needed to help 
teachers use data more effectively. The survey and its results led to the development of a Teacher 
Data Use Survey tool for broader use among alliance members and their colleagues, as well as a 
webinar series on how to implement and report on the survey. 


e Using locally based staff to help build capacity. Particularly in geographically distant regions, RELs 
sometimes used locally based REL staff, consultants, or experienced practitioner alliance members 
to assist alliances in providing more hands-on and ongoing technical assistance to alliance 


members; this support was sometimes needed to ensure that the research agenda stayed on pace. 


Other strategies used by RELs to support research alliances 


As discussed throughout this chapter, RELs engaged in a broad array of strategies to address the challenges 
that they faced in supporting the development of research alliances. In addition, RELs engaged in the 
following strategies that could be relevant for multiple challenges that they faced in supporting their 


research alliances: 


e Capitalizing on the diversity of skills within the RELs when assigning specific REL staff members 
to work on as-needed alliance-related projects. Because different REL research alliances had 
different needs at different times, the RELs designated appropriate staff to meet the needs of the 
particular alliance for building capacity and developing and carrying out research agendas. For 
example, during earlier stages when an alliance was developing a plan for a research study, REL 
staff who had research expertise were needed to support framing a research design and determining 
appropriate analytic methods. However, if a research alliance’s priority at another stage was more 
focused on capacity building, then RELs would designate staff with expertise in providing technical 


assistance and support. 


e Engaging in cross-alliance support activities. In cases where different research alliances within a 
REL had a shared problem of practice or an issue that they were exploring, the RELs were able to 
coordinate technical assistance or dissemination efforts across alliances. For example, RELs engaged 
in cross-alliance webinars on specific topics, including, in a few instances, webinars focused 


specifically on best practices that support the development of research alliances. 
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e Providing opportunities for REL researchers to develop their skills in working with 
practitioners. RELs reported providing internal professional development to their research staff for 
understanding issues related to doing collaborative researcher-practitioner work. This ranges from 
the formal creation of a professional learning community or training of locally based consultants, to 
less formal mechanisms such as brainstorming sessions, monthly check-ins, planned “water cooler” 
conversations held throughout the year, or through providing staff with useful literature and 


resources related to team-building, engagement, and research-practice partnerships. 


Providing opportunities to develop skills to bridge the researcher-practitioner divide: an example 
from REL Midwest 
REL Midwest supported the development of a professional learning community of REL researchers 


working on research alliance-related projects. This group discussed issues related to engaging in 
collaborative research with practitioners, breaking down silos between researchers and 
practitioners, and using time efficiently and effectively. For example, they documented topics of 
conversation that occurred between researchers and stakeholders and discussed optimal 
communication strategies that should be considered at each stage of a research project. 


e Participating in cross-REL activities related to the research alliances. By virtue of working within 
the larger network of the national REL program, RELs had opportunities for cross-REL 
communication and collaboration. IES required each REL to designate one member of its 
leadership team to lead and oversee all alliance efforts; these individuals worked with other internal 
REL staff and researchers to serve the alliances and also collaborated with their counterparts across 
the other RELs. In particular, the 2012-17 REL program included cross-REL working groups 
meant to facilitate information sharing and collaboration across RELs; one working group was 
focused on research alliances.*° This working group, consisting of the alliance lead staff from each 
REL, met quarterly to discuss shared experiences and challenges and to make plans for the 
development of cross-REL products or presentations (for example, presentations at national 
conferences). This collaboration enabled RELs to learn from the experiences of other RELs and to 
better serve the membership of their own alliances. Additionally, all REL directors met biannually 
through meetings that IES convened, often discussing the progress of research alliance efforts and 


sharing best practices. 


*° See the “Performance Work Statement for Regional Educational Laboratories (2012-2017)” available at 
https://www.tbo.gov/utils/view?id=50688bfda3cc31d77469838a7964949 1. 


37 


Conclusion: Implications for networks of research-practice partnerships 


Just as past research and work on research-practice partnerships has helped inform the development and 
implementation of REL research alliances, so too can the REL experience inform current and future 
research-practice partnerships. The 2012-17 REL research alliance experience is particularly useful to 
consider, given the wide variety of alliances involved and the scale of the effort. Alliances came in many 
shapes and sizes and varied in structure, topics covered, membership, and geographic spread. Chapter 4, 
in particular, highlighted the variety of challenges that REL research alliances faced and the specific ways 
that REL staff attempted to deal with these challenges (summarized in table 3). 


While some of these challenges and identified strategies were specific to the REL experience, many of the 
methods that research alliances used to address challenges can be applied to a wider set of research- 


practice partnerships. 


Implications for building trust, engagement, and a sense of partnership. Research-practice partnerships 
may want to consider leveraging existing relationships, encouraging practitioner members to take on key 
leadership positions, and thinking about how participants can take on differentiated roles to foster 
ownership and engagement. The REL experience also points out the usefulness of engaging staff members 
to support the alliances who have a deep understanding of both the researcher and practitioner 


perspectives. 


Implications for fostering communication. The RELs acknowledged that early and ongoing 
communication was critical in engaging and supporting research alliances as they evolved, which is 
relevant to any research-practice partnership. The REL experience shows the value of using a myriad of 
methods to communicate with research-practice partners and other key stakeholders especially when they 
are spread across multiple communities. This includes both formal and informal communication, using 
in-person and virtual methods and using a variety of different media such as reports and briefs, 


presentations, archived videos, and infographics. 


Implications for managing time constraints. Research-practice partnership members always have other 
responsibilities and time constraints that can limit participation. RELs found that it was important to 
consider time demands when determining who could take on key leadership roles. Also, they found that 
continual recruitment can help combat membership attrition due to time constraints. The REL research 
alliance experience also points out the importance of thinking about realistic timelines that meet the 
needs of stakeholders and ways to creatively deal with time lags in obtaining research evidence by, for 
example, providing other valued products or services such as sharing materials on emerging evidence or 


providing targeted technical assistance. 


Implications for working with research alliance members who have different priorities, interests, and 
expectations. The REL research alliance experience shows that size of the research-practice partnerships 
and geographic (and jurisdictional) composition have strong implications for research agendas and should 
be considered early on and continually reconsidered as research-practice partnerships progress and evolve. 


Because state and local personnel change and district and state priorities change, the RELs found that 
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their research alliances needed to be nimble and flexible. This may require subdividing groups or creating 
smaller working groups, combining groups, or recruiting new members. As with any research-practice 
partnership, the RELs found that research alliance members needed to find common ground by 
collaboratively developing an agenda around a common set of interests and engaging in regular reflective 


work to ensure that this evolving agenda remained relevant. 


Implications for building capacity. The REL program’s structure facilitated multiple layers of support for 
the development of research alliances, which other research-practice partnerships may want to consider. 
RELs used specific staffing, internal structural strategies, and procedures to take advantage of their 
organization’s capacity to support REL research alliances and to develop the capacity of research alliance 
members. This included the development of toolkits, providing targeted technical assistance, and engaging 
locally based staff to support research alliance members. The structure and requirements of the overall 
REL program also provided opportunities for cross REL communication and support. While the activities 
that RELs used were facilitated by the structure of the REL program, other networks of research-practice 
partnerships can learn from these experiences. In particular, research-practice partnership networks may 
want to consider how an umbrella coordinating entity can be formally structured to help identify and 
support broad needs and facilitate opportunities for collaboration and dissemination of lessons learned 


across various partnership efforts. 


The REL 2012-17 contract cycle was responsible for the implementation of the largest research-practice 
partnership effort in preK-12 education to date. This effort spanned all regions under jurisdiction of the 
U.S. and territories and included more than 70 distinct research alliances at any point during the 5-year 
contract cycle. RELs relied on (1) strategies endorsed within the general literature on education research- 
practice partnerships within public education, (2) the collective research experience within the REL 
program and IES, and (3) innovative solutions from merging these two bodies of knowledge and practice 


to address each major challenge encountered in their efforts to establish and support research alliances. 


Research-practice partnerships vary considerably in scale and scope, but at the heart, share the common 
goal of building the knowledge base to support the work of educators. As research-practice partnerships in 
education continue to grow in popularity, it will be important to continue to document these experiences, 


understand their commonalities and differences, and share lessons learned. 
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Appendix A. The 79 REL program research alliances by REL (January 2015) 


REL Appalachia REL Central 
e Kentucky College and Career Readiness Alliance e Educator Effectiveness Research Alliance 
e Metro Nashville Public Schools Data Use Research Alliance © Formative Assessment Research Alliance 
e Virginia Middle School Research Alliance e@ Mathematics and Science Partnership Research Alliance 
e West Virginia Workforce Readiness Alliance @ Native American Education Research Alliance 
e West Virginia School Leadership Research Alliance ® College and Career Readiness Alliance 
e@ Appalachia Higher Education Consortium e Rural Education Research Alliance 
e Systems Development and Improvement Research Alliance 


REL Mid-Atlantic REL Midwest 


e Early Childhood Education Research Alliance Beating the Odds Research Alliance 
®@ Mid-Atlantic Historically Black Colleges and Universities College and Career Success Research Alliance 
College Completion Research Alliance Dropout Prevention Research Alliance 


Longitudinal Data Use Research Alliance Early Childhood Education Research Alliance 
Principal Effectiveness Research Alliance 


Professional Learning Research Alliance 

Rural Student College Readiness Research Alliance 
School Completion and Engagement Research Alliance 
Teacher Evaluation Research Alliance 

REL Northeast & Islands REL Northwest 


Rural Research Alliance 
Teacher Effectiveness Research Alliance 
Urban Research Alliance 


e 
e 
e 
e 
e 
e 
e 
e@ Virtual Education Research Alliance 


e Northeast Educator Effectiveness Research Alliance e Alaska Statewide Policy Research Alliance 
e Northeast Rural Districts Research Alliance @ Idaho System of Recognition, Accountability, and Support Alliance 
@ Urban School Improvement Alliance e Montana Data Use Alliance 
e@ Early Childhood Education Research Alliance ® Oregon College and Career Readiness Research Alliance 
e English Learners Alliance @ Oregon Leadership Network 
e Northeast College and Career Readiness Research Alliance @ Washington Educational Service District Network Alliance 
e Puerto Rico Research Alliance for Dropout Prevention e Road Map for Education Results Alliance 
e U.S. Virgin Islands College and Career Readiness Research @ Tribal Educators Alliance 
Alliance 
REL Pacific REL Southeast 
e@ Hawai'i Partnership for Educational Research Consortium Improving Low-Performing Schools in Alabama Research Alliance 


(Optimizing Data Readiness in Hawai'i) 

Yap Research Alliance 

Chuuk Research Alliance 

Pohnpei Research Alliance 

Kosrae Research Alliance 

Marshall Islands Research Alliance 

Palau Research Alliance 

Guam Alliance for Family and Community Engagement 
American Samoa Alliance for College and Career Readiness 


Commonwealth of the Northern Mariana Islands (CNMI) 
Alliance for College and Career Readiness 


REL Southwest REL West 

California Office to Reform Education (CORE) Alliance 
Community College Alliance on Career/Technical Education 
Dropout Prevention Alliance for Utah Students with Disabilities 
Educator Effectiveness Alliance 

English Learner Alliance 

Middle Grades School Climate Alliance 

Nevada Education Research Alliance 

Silicon Valley Research Alliance 


Improving Schools in Mississippi Research Alliance 
Improving Literacy Research Alliance 

Improving Mathematics Instruction Research Alliance 
Georgia Charter School Alliance 

School Leadership in North Carolina Research Alliance 
School Readiness Research Alliance 


e 
e 
e 
e 
e 
e 
e 
@ Blended and Online Learning Research Alliance 


Arkansas PreK Research Alliance 

Louisiana Charter School Research Alliance 

New Mexico PreK Research Alliance 

New Mexico Achievement Gap Research Alliance 
Oklahoma Rural School Alliance 

Texas Hispanic STEM Alliance 

Educator Effectiveness Research Alliance 
English Learners Research Alliance 
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Appendix B. The evolution of REL research alliances from year 1 to year 4 


Because the REL research alliance effort was a new and groundbreaking initiative for the RELs and 
because it was implemented on a scale beyond any other research-practice partnership efforts, it was 
unclear how alliances would change over time. In general, the transformations that REL research alliances 


experienced from year | to year 4 occurred for three main reasons. 


Priorities changed over time. RELs conducted needs assessments and worked with their research alliance 
stakeholders to prioritize goals and to develop research agendas. The priorities of those initially proposed 
goals may have been adjusted over time as needs changed. Changing priorities resulted in the creation of 
new alliances and the transformation of existing alliances. In other cases, the composition of alliance 


membership changed over time, and newer members had different priorities and goals. 


Priorities were broad or varied. A majority of the REL research alliances that originated during year 1 
were newly created and brought together members from different disciplines, districts, or states for the 
first time. These stakeholders may have had different priorities, goals, and interests. In some cases, for 
example, REL alliance members realized that their needs and priorities were too broad or too varied and 


would be better served by separate alliances (for example, alliances for specific geographic regions). 


Priorities evolved as work progressed. Alliance research and technical assistance projects produced 
lessons learned and suggested next steps that helped to reframe priorities, thus allowing REL research 


alliances to evolve naturally over time. 


While not all REL research alliances changed substantively over time, many did. Among the 58 alliances 
that were in partnership with a REL in both year 1 and year 4, 22 did notsubstantively change over time. 
However, 36 alliances did change their name or geographic coverage or were split or consolidated.” 


Alliance transformations were manifested in the following three ways:” 


e Fora variety of reasons, 29 alliances changed their names.*’ In some cases, REL research alliances 
changed their names to clarify or be more reflective of the work in which they were engaged. In 
other cases, alliances changed their names to reflect modifications over time in substantive focus or 
geographic coverage. For example, REL Northwest’s Montana AA District Network Research 


3! This conception of “transformation” does not reflect changes in REL research alliance membership, which was often in flux 
over the course of multiple years across alliances. Thus, some REL research alliances that may not have changed in terms of 
geographic focus or substantive focus may have experienced large changes in terms of individual membership over the 4-year 
time span. 

*? REL research alliances may have transformed in multiple ways. For this reason, the totals across the three categories do not 
add up to 36. 

* Alliances that changed their names only slightly (for example, by adding “research” in front of “alliance” or changing “state” 
to “statewide”) are not included in this count. This count includes name changes due to modifications in focus, geography, or 
structure. 
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Alliance, which originally started as a seven-school district partnership, expanded statewide and 


changed its name to the Montana Data Use Alliance. 


e At least 23 alliances modified their geographic coverage. As REL research alliances evolved over 
time, 13 expanded their membership across more states, 8 reduced the number of states involved, 


and at least 2** substantially broadened their reach within a state. 


e Seven of the alliances active in 2015 were the result of either a split or a consolidation of alliances 
from their original configuration in year 1. Six split into two or more alliances, and one 
consolidated multiple alliances into a single entity. For those that split, the REL staff and research 
alliance members decided that their efforts would be better served as separate entities. For example, 
REL Pacific chose to break the College and Career Readiness Alliance into two alliances—one 
focusing on American Samoa and the other on the Commonwealth of the Northern Mariana 
Islands. In another example, REL Mid-Atlantic’s initial Teacher Effectiveness Research Alliance 
broke into two alliances—one focusing on teacher evaluation and another focusing on professional 
learning. In contrast, REL Central chose to consolidate four alliances related to teaching and 
professional development into workgroups that operated under the umbrella Educator 


Effectiveness Research Alliance. 


All but one REL had at least one research alliance that transformed in some way by 2015, and there was 
wide variation across the RELs (figure B1). In 6 of the 10 RELs, at least half of the 2015 alliances had 
transformed since year 1. Six RELs had at least one alliance that changed its name. Seven RELs had at 
least one alliance that modified its geographic coverage. Five RELs had at least one alliance that split or 
consolidated. Seven RELs added at least one new alliance since year 1. For only two RELs, the majority of 


the alliances in year 4 were the same as those in year 1 with no substantive changes. 


* Tris possible that more REL research alliances expanded within a specific state. Our count is based on information 
documented in each REL’s discussion of its alliances in the annual plans submitted to IES. The RELs were not required to 
provide this level of detail in their plans. 
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Figure B1. REL research alliances in 2015 were more likely to be modified versions of 2012 alliances than 
unchanged or newly created; but changes over time varied widely by REL 


ray 
(o) 


Number of alliances 
OrPFN Ww FSU DN WO OO 


@ No change & New alliance 
Transformed name # Transformed geographic coverage 


#2 Transformed by splitting or consolidating 


Note: This figure reflects only those REL research alliances that were in partnership with a REL as of year 4. It does not include the 15 REL research 
alliances that were active in year 1 but were no longer in existence or in partnership with a REL as of year 4. In some cases, REL research alliances 
transformed for multiple reasons. This chart documents the predominant reason for the change, prioritizing changes in the following order: 
splitting/consolidating, geographic coverage, and name. For example, if an alliance changed its name to account for a geographical expansion, it 
would be categorized in this figure as “Transformed geographic coverage.” 


Here, we have discussed the evolution from year | to year 4 in terms of broad issues—specifically, the 
creation of new alliances and the closure or modifications of existing alliances. In reality, regardless of the 
consistency of their original name and structural configuration, alliances may have changed substantially 


in membership composition and research focus over time, as discussed in chapter 4. 
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